Stein ’ s Unbiased Risk Estimate Statistical Machine Learning , Spring 2015

1 Stein’s lemma • In a landmark paper, Stein (1981) derived a beautiful and simple lemma about the standard normal distribution. Indeed, Stein knew of this result much earlier and wrote about it in previous papers, but in Stein (1981), the author developed a multivariate extension of this lemma that led to a remarkable result on unbiased risk estimation. (And, an interesting note: the paper Stein (1981) itself was actually written in 1974, and rumor has it Stein wasn’t planning on publishing it, until a colleague convinced him to do so in 1981...) • We’ll walk through Stein’s univariate and multivariate lemmas on the normal distribution. Following this, we’ll discuss how they apply to unbiased risk estimation. We note that the univariate lemma has a converse, and this has become extremely important in its own right, studied and further developed in probability theory for proving convergence to normality. Stein didn’t write a lot of papers, but he was a pretty influential guy!